Data processing systems are well known in the art. Conventionally, large amounts of data are stored in relational databases. Relational databases organize data into a plurality of tables and include mechanisms to facilitate efficient querying, retrieving, and data processing. In particular, tables are partitioned into rows and columns such that a value exists at the intersection of each row and column. Each column typically contains values of the same data type. For instance, a sales table can consist of columns for the name of the purchaser, the product name, the price, the location, etc. The rows contain values for each of these columns. Data can be retrieved from and provided to tables utilizing structured query language (SQL) statements to communicate with a database server management system (e.g. relational database management system (RDMS)). Accordingly, one can specify a query and receive a set of results in response thereto (e.g., a table of sales data). When the database is quite substantial in size, such processing of data is often referred to as on-line transactional processing (OLTP). For instance, airlines and hotels can utilize OLTP to process reservations and banks as well as automated teller machines (ATMs) can utilize OLTP for electronic fund transfers.
There are several keys to optimizing conventional database transactions. First, queries must be satisfied quickly without blocking others from accessing data. Relational databases are typically set up and utilized by a myriad of people. Hence, when data is being accessed from a database to satisfy a query or otherwise execute a SQL instruction, the database is locked to guarantee data integrity. In other words, concurrent users of the data must wait to execute their queries until the current action is terminated. Accordingly, it is desirable to execute queries expeditiously to minimize others waiting period. Second, the database must provide a reasonable size of data back to the user as soon as possible so that something can be done with it. For instance, one might query a database to generate a report, perform some calculation or other data processing. In essence, conventional data processing consists of getting in and out of the database as fast as possible to ensure its availability to others.
By contrast, applications require very rich interactions with data. For example, in a typical email application, users desired to scroll up and down through data as well as jump to and from different points in the data. Furthermore, users desire to group related data and sort based on the data members and the grouping of databases. Still further yet, users desire to interact with such data in a very live an interactive fashion. Thus, there is a tension between applications and shared data. That is, applications desire to interact very richly with the data while stores require very well defined interactions with the data that are very strictly scoped in terms of time and amount of data. This can become particularly problematic when confronted with a large amount of data or a potentially large amount of data. Conventionally, what an application will do to enable a user to scroll over say twenty different records is that it will query those records, store them on the client, and then release the database connection. The user can then scroll through the data on the client. Additionally, the user can make changes to the data on the client and then possibly push those changes back to the database. Unfortunately, the conventional paradigm breaks down when you have large amounts of data in part because it is not desirous to have to cache large amounts of data on a client machine. For example, if a user executes a query or search and receives 15,000 hits, it is quite inefficient and unreasonable to read and store all 15,000 hits onto the client's machine before allowing them to scroll through the first three or four that might be of the most interest. In such a case, it might be predictable which data will be of interest, namely the first three or four. However, in other applications, such as email, one could be interested in any data throughout the entire result set. For example, one might specify a query to show all email where certain conditions are met (e.g., received is me). In that case, the user will want to scroll through all the data, but they will not want to store it all on their client device.
Moreover, users want the interaction with data to be very live (e.g., in real-time) so as changes happen they are reflected in the data. For instance, if new data arrives they want to see that data and as data is deleted they want to see such data removed. For instance, when an email goes from the state of unread to read, the displayed mail should go from bolded to un-bolded almost instantaneously. Also, as new mail comes into a user mailbox they want to see it appear. These and other types of rich applications are not supported well by relational databases.
In order to allow rich applications to work in concert with relational databases, as described more fully below, application views are used to provide rich interaction with persisted data. In particular, an application view is generated for each application and dynamically maintained to provide a rich experience working with data, including but not limited to an ability to scroll through large sets of data while viewing additions, deletions, and other changes made to the underlying data in real-time.
In one implementation, rich applications define the set of results to be maintained for use by the application by specifying:                the source,        the set of projected columns or expressions, and        filters, sorts, aggregations and groupings.This may be done by creating ViewDefinition objects and setting properties on such objects, but other means of expressing these query components, including, e.g. parsing a query syntax, are equally valid.        
In order to support rich views over these results, in one implementation, temporary tables are generated populated with the results of the query. Populating these temp tables can be a data intensive and long running query depending on the specification of the query supplied. As a result the initial load time will adversely impact the time from the creation of the view to when results are available to the application. Extensive performance analysis has revealed that the initial population of the view is the main hotspot and bottleneck in terms of performance and where the majority of the processing time takes place during the lifetime of the view.
Thus, a user may experience a delay during the use of an application with a rich application view. Such a delay may adversely affect the user experience.
In view of the foregoing, there is a need for systems and methods that overcome such deficiencies.